Faces and Speech: Intermodal Processing of Biologically Relevant Signals in Infants and Adults

The human face stands out as the single most important stimulus that we must recognize in the visual domain. In the auditory domain the human voice is the most important hiological signal. Our faces and voices specify us as uniquely human, and a challenge in neuroand cognitive science has been to understand how we recognize and process these two hiologically relevant signals. In hoth domains the conventional view is that the signals are at first recognized through unimodal mechanisms. Faces are thought to he visual objects and voices to be the province of audition. We intend to show that these stimuli are aru~lyzed and represented through more than a single modality in hoth infancy and adulthOod. Speech information can he perceived through the visual modality, and faces through proprioception. Indeed, visual information ahout speech is such a fundamental part of the speech code that it cannot he ignored hy a listener. What listeners report "hearing" is not solely auditory, hut a unified percept that is derived from auditory and visual sources. Faces and voices are thoroughly intermodal ohjects of perception. Recent experiments have discovered that infants code faces and speech as intermodal ohjects of perception very early in life. We focus on these intermodal mappings, and explore the mechanism hy which intermodal information is linked. Faces and speech can he used to examine central issues in theories of intermodal perception. How does information from two different sensory modalities mix? Is the input from separate modalities translated into a "common code?" If so, what is the nature of the code? One phenomenon we discuss is infants' imitation of facial gestures. Infants can see the other person's facial movements hut they cannot see their own